Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Python named entity recognition model based on transformer
Guanyou XU, Weisen FENG
Journal of Computer Applications    2022, 42 (9): 2693-2700.   DOI: 10.11772/j.issn.1001-9081.2021071356
Abstract423)   HTML34)    PDF (1723KB)(194)       Save

Recently, some character-based Named Entity Recognition (NER) models cannot make full use of word information, and the lattice structure model using word information may degenerate into a word-based model and cause word segmentation errors. To deal with these problems, a python NER model based on transformer was proposed to encode character-word information. Firstly, the word information was bound to the characters corresponding to the beginning or end of the word. Then, three different strategies were used to encode the word information into a fixed-size representation through the transformer. Finally, Conditional Random Field (CRF) was used for decoding, thereby avoiding the problem of word segmentation errors caused by obtaining the word boundary information as well as improving the batch training speed. Experimental results on the python dataset show that the F1 score of the proposed model is 2.64 percentage points higher than that of the Lattice-LSTM model, and the training time of the proposed model is about a quarter of the comparison model, indicating that the proposed model can prevent model degradation, improve batch training speed, and better recognize the python named entities.

Table and Figures | Reference | Related Articles | Metrics